Agile testing: Should you disband the test team?

The happy path
Before exploring the possible difficulties of Agile, let us describe the 'happy path' of Agile methods. Agile is best suited to small to medium sized development projects where the developers, testers and the customer are all under one roof. The development work is broken down into 'sprints' of two to four weeks, each of which will deliver some working code. The testers are no longer organised as a separate team which tests finished modules of code. Instead they are embedded with the developers and continually test small pieces code as soon as they are written.
The testers implement a high degree of automation in the compilation and testing of code, so that the whole code set is compiled every evening and automated unit tests run on it. The compilation and running of the tests is typically organised by a tool like Hudson or Jenkins. The automated tests are written in tools like JUnit, Selenium and FitNesse.
Methodologies like test-driven development (TDD) are common, and tests are often written before the code. Testers are pro-active; they sit close to the developers and take part in the daily scrum meetings (or equivalent). They also liaise often and closely with the customer team to understand the customer's requirements (stories) as soon as possible and translate them into tests. A traditional tester, by contrast, sits in a silo writing tests based on a functional or non-functional specification document. As code is thrown over the wall from the development team to the test team, the tester tests it and sends any defects back to the developers.
When all the defects have been fixed and retested, the test team declares the code passed and issues a certificate to that effect. Work does not begin until the entry criteria are met, and does not stop until the exit criteria are met. Agile testers are expected not to conform to this mindset and instead be pro-active in dealing with the developers and the customer. Ideally, this speeds up testing and finds bugs more quickly.

Disbanding the test team
Unfortunately, not all projects follow the Agile happy path. Agile methods were conceived primarily for software development. A great many IT projects, however, do not involve development. A huge amount of the time and resources of most IT departments is taken up with infrastructure upgrades, migrations and systems integration, rather than development.
An IT department which disbands its test team may have problems organising its testing when it comes to a non-development project. In the absence of developers, the testers must come up with a set of functionality to be tested and a way of documenting the tests and results, and there is little alternative but to follow a traditional pattern.
A project board is unlikely to be comfortable in approving the go live of, say, a Citrix upgrade unless it can see evidence of testing in a traditional test completion report. Tools which are common in Agile development, such as Jenkins and JUnit, are simply irrelevant in a migration project. Some Agile testing techniques, however, can be deployed, especially the pro-active mindset and will to pre-empt problems, but the test methodology is likely to be more traditional.

Scale & distance
Agile methodologies also encounter problems with scale and distance. Ideally, an Agile project team is located under one roof and meets daily in a scrum or similar meeting. Stories are written on 'post-its' and stuck onto a board which the whole team can see. As stories are written, worked on and completed, the post-its are moved across the board to show progress. Following this methodology is not so easy if the project is large and requires hundreds of developers.
It is even harder if the developers and testers are spread across several continents. Here project and test management tools become essential. Many tools reproduce the story board electronically, such as Pivotal Tracker, the Rally suite and HP Agile Manager. Some integrate it with bug tracking software to produce an integrated project and quality management tool. For large Agile projects, the development is often broken down into several different Agile teams. Here the challenge is to co-ordinate the activities of the different teams. The risk is that dependencies are created between teams and that with less formal documentation, one team does not understand the outputs produced by another. Again, management tools can help. As regards testing, a defect management tool is especially useful with multiple Agile teams working on the same project.

The challenge of regulation
Another challenging area for Agile testing is in regulated industries such as pharmaceuticals, finance and aerospace. In these industries, regulatory bodies often require formal evidence of testing. If badly managed, software which has been developed using Agile methods must then go through a formal test phase to show that it complies with applicable regulations.
More traditional development methods lend themselves to producing the documents required by the regulators. To meet regulatory requirements without losing the benefits of Agile, using a suitable test management tool can help. HP has adapted Quality Center over the years so that it now caters for Agile projects, and newer tools such as TestWave do so as well. The tools can support Agile development sprints but can also draw out all the tests and results needed for a formal report to regulators.

Agile people
Agile works best on small and medium sized development projects and is less well suited to non-development projects, large and dispersed projects and heavily regulated industries. So should a CIO disband the test team? It depends on a number of factors. How much of the department's projects consist of development? How big are the projects? With how much outside regulation does the department have to comply? Ideally an IT department should keep some dedicated test expertise to determine the test strategy for each type of project the department undertakes, and then organise the testers appropriately, with projects having separate test teams or not depending on the type of project.
Even in an Agile project with no separate test team, keeping some reporting lines to a test manager outside the various projects is a good way maintain testing specific skills, and to benefit from the resulting product quality. More fundamentally, the appropriate handling of Agile testing comes down to people. If you hire the right people, they will thrive in Agile projects (which require more initiative and flexibility from testers than traditional methods) and have the awareness to see where a different approach is required. Tools can also help by bridging long distances and simplifying reporting.